Welcome to the second part of this discussion on
heterogeneous integration. In this video, I'm
going to continue the discussion on Roadmapping heterogeneous integration
for interconnects and try and describe a future. Now that we've learned the basics of how
to do roadmapping. Let's first ask the question, why should we be doing
interconnect scaling? The answer is pretty simple. Generation over generation, the speeds of the links that leave a package have increased. They have roughly double about every two and a quarter years and they want to stop at
least in the near future. More importantly, the
amount of bandwidth needed has continued to increase as applications have started
to use more and more data. These two graphs describe to you how the byte to flop ratio, which means the memory
to compute ratio, the demand has
evolved over time, and this in my mind, is a
very interesting graph. What this graph says
is that over time, the bytes to flop
ratio capability has gone down because compute has
moved faster than memory. In an ideal world, if you were able to increase
the bandwidth for compute by improving the connectivity
between different dye, you can get considerably
higher amount of performance. There's a couple of interesting
things about this graph. Compute has evolved, memory bandwidth
has not kept up. But in the year 2018, when each BM memory interconnect started to become mainstream. You will notice
that this curve has seen a slight inflection, which means you can get higher and higher
performance if you are able to feed compute with
higher bandwidth data. The goal for packaging is to
try and push this curve up. The two conclusions that
must come out of the slide. There is an increasing
or growing demand for bandwidth and in order to be able to supply
this bandwidth, you need dense
interconnects that are designed to be low power. Now if we take this
logic argument, I have talked about
the 2D interconnects and the 3D interconnects. As I mentioned before, stick only to solve their best interconnects
and you ask yourself, how am I going to increase the amount of bandwidth and
start creating a roadmap. Let's define a generation where going from one
generation to another, you double the raw
bandwidth density as measured by gigabytes
per second per millimeter. You start with HBM, which had a pitch of 55 microns, had IO per millimeter capability
of about 500 wires per millimeter and if you look at the bumps per millimeter squared or IO per
millimeter squared, 331, and you send signals
at 2 gigabytes per second. You get 125 gigabytes
per second. I should point out that
this is a non-unique curve. As you can split the IO
per millimeter square, and this splits differently. But so this is an example
of how you get them. If you now wanted to
double the bandwidth as you wanted to go
from 125 to 250, what you would do
is you could shrink the pitch a little, go to 50. You can shrink the
IO per millimeter, go to 667, increase the
signaling speed a little bit, you have doubled the bandwidth. Going through the same logic
and if you want to stay with solder based interconnects
and you say I draw the line at
30 micron pitch, you can go for at
least five generations and continue to
double the bandwidth. There is no reason why
this graph cannot move to the right because you
can increase the speed, you can increase the
bump pitch slightly. You could change the
IO per millimeter. This roadmap can be
extended to the right. Notice however, that the
signaling speed when you use solder based interconnects
has to increase, and it is known that as
you increase the speed, you also increase the power. Now if you say you want to change the physical
interconnects, but you don't want to
increase the speed so much and you go on to transition from solar
based interconnects to the hybrid bond interconnect, so the copper covering
techniques I talked about, you can achieve this. You can stay at a
signaling speed of four gigabytes per second. You can shrink the pitch
aggressively going from 55 to 10 and 10 is nowhere close to a limit
because as I said before, you have demonstrated
performance of nine micron pitch already. You can still achieve
bandwidth doubling. You have multiple choices in increasing bandwidth
generation over generation by changing
the bump pitch, by changing the wires per millimeter and
changing the speeds. What this table says is, you have an opportunity to
continue to scale bandwidth. You have different
choices you make when you scale bandwidth for
2D interconnects. We talked about 2D interconnects or
lateral interconnects. Now let me talk about
3D interconnects. Same bandwidth targets, 125, 250, 500,1000 and 2000. Here, if I start with
a pitch of 40 microns, which has been implemented
in a real product, I can achieve a bump or IO
per millimeter square of 625, and at a signaling speed of
1.6 gigabytes per second, I can hit 125 giga. Signaling speed of 1.6
gigabits per second, I can achieve a
bandwidth density of 125 gigabytes per second
per millimeter squared. I can continue to aggressively
shrinks the pitch. Go down to 10 microns, which is no limit of
any kind because you can actually shrink pitch
much lower than this. You can continue to double
that bandwidth generation over generation and you can do it
at lower signaling speeds because you get an area
advantage when you do 3D sticks. The other aspect of all
of this is that you must have some standards
when you connect this, and the good news is
that a standard called UCIe has been released
by a consortium. I should point out
that there are other standards as well. I'm using this as a
example standard. The standard is done
by a consortium of people coming together
to standardize the interconnects so that
different devices can be stitched using the
same standard. The standard describes the
key performance metrics and it allows you to do
what is true plug-and-play. The best way to
understand plug-and-play is to understand how
LEGO blocks work. If you created LEGO
blocks that can be stitched to each other
and by that same analogy, if you create die which can be stitched to each other
using standard interfaces, you can build out a network of connectivity on a
package and you can get performance that
can be tailored depending on the
number of die and the characteristics of the die. UCIe is a good standard that allows you to do this and
it allows different people to conform to the
standard and create the characteristics
desired for performance. These characteristics
and key metrics are described in this graph. The UCIe 1.0 spec describes the different
data rates possible, the different clusters
that can be built, the reach that you can
go and traverse across. It describes the
shoreline density as in how much bandwidth you
can get across a die age. It describes the aerial
density you can get. It also describes a key metric called the power
efficiency target, as in how much energy is
spent in transmitting a bit because you're per bit and it shows you that you
can scale it down in time. It describes another
aspect which I have not discussed before, latency. It tells you the latency of the signals that go across
the interface as well. The goal behind the
UCIe is standard and the goal behind other
standards as well is to align the industry towards common standards so that
we can collectively and cohesively drive improvements in heterogeneously
integrated packages. So far I have talked about
die to die techniques, whether they are
lateral or vertical. But it is also important to
understand that you have to deal with signals
that go off the package, and a very, very powerful
interconnect technology is optical interconnect
technology. If you look at this
table on the right, you will notice that there are different attributes of optical that are pretty impressive. For instance, if I look
at a comparison on a copper-based signaling
techniques like PCIe Generation 6 and
compare it to optical, with PCIe, I can get about 380 gigabytes per second
per millimeter of diage. With optical IO, I can get
800 and I can double it to 1.6 terabytes per second. You get a 4x or 4 times
Beta interconnect. The reach of optical is larger, so you can get
greater than 100x. You can get roughly
equivalent latency with careful design. You have an opportunity to
get 35 percent improvement in power efficiency and you can get greater and
greater bandwidth density. The way to think about
optical interconnects is first think of
optical interconnects as package to package
interconnect, and as a way to
complement copper, co-package optics will
complement copper and not essentially always
replace copper. There have been quite a
few demonstrations of optical co-package optics
improving bandwidth density. On the top left there is a demonstration of an Ethernet
switch and the bottom right there is an
FPGA integrated with an optical tile to do top
of package optical links. This is a technology
that is coming in the sense you
need these densities for improved performance
and its maturity will depend on how effectively we are able
to drive the cost down, how effectively
variable to get to high yielding
mature technologies and a supply chain
that supports it, and how effectively
we are able to manufacture these at
high yield to cut cost. These are key challenges that
are being worked on today. In conclusion, I have described how you can
scale 2D interconnects. I have described
how you can scale 3D interconnects that will allow you to do continued
bandwidth scaling. Another part of this
is you standardize the link to create the
plug-and-play infrastructure. This will allow systematic
growth in the industry, and co-packaging optics is a key element of future of
package bandwidth scaling. These three vectors
will be very, very important to
continue to evolve on to get greater and
greater performance.